Logistic Regression

Logistic regression is a predictive modelling algorithm that is used when the Y variable is binary categorical. That is, it can take only two values like 1 or 0. The goal is to determine a mathematical equation that can be used to predict the probability of event 1. Once the equation is established, it can be used to predict the Y when only the Xs are known.

Earlier you saw what is linear regression and how to use it to predict continuous Y variables.

In linear regression the Y variable is always a continuous variable. If suppose, the Y variable was categorical, you cannot use linear regression model it.

So what would you do when the Y is a categorical variable with 2 classes?

Logistic regression can be used to model and solve such problems, also called as binary classification problems.

A key point to note here is that Y can have 2 classes only and not more than that. If Y has more than 2 classes, it would become a multi class classification and you can no longer use the vanilla logistic regression for that.

Yet, Logistic regression is a classic predictive modelling technique and still remains a popular choice for modelling binary categorical variables.

Another advantage of logistic regression is that it computes a prediction probability score of an event. More on that when you actually start building the models.

Building the model and classifying the Y is only half work done. Actually, not even half. Because, the scope of evaluation metrics to judge the efficacy of the model is vast and requires careful judgement to choose the right model. In the next part, I will discuss various evaluation metrics that will help to understand how well the classification model performs from different perspectives.

You might wonder what kind of problems you can use logistic regression for.

Here are some examples of binary classification problems:

When the response variable has only 2 possible values, it is desirable to have a model that predicts the value either as 0 or 1 or as a probability score that ranges between 0 and 1.

Linear regression does not have this capability. Because, If you use linear regression to model a binary response variable, the resulting model may not restrict the predicted Y values within 0 and 1.

Logistic regression achieves this by taking the log odds of the event ln(P/1?P), where, P is the probability of event. So P always lies between 0 and 1.

Taking exponent on both sides of the equation gives:

Hi! I am Selva, and I am excited you are reading this! You can now go from a complete beginner to a Data Science expert, with my end-to-end free Data Science training. No shifting between multiple books and courses. Hop on to the most effective way to becoming the expert. (Includes downloadable notebooks, portfolio projects and exercises)

Start free with the first course 'Foundations of Machine Learning' - a well rounded orientation of what the field of ML is all about.

Enroll to the Foundations of ML Course (FREE)

Sold already? Start with the Complete ML Mastery Path

You can implement this equation using the glm() function by setting the family argument to "binomial".

In above model, Class is modeled as a function of Cell.shape alone.

But note from the output, the Cell.Shape got split into 9 different variables. This is because, since Cell.Shape is stored as a factor variable, glm creates 1 binary variable (a.k.a dummy variable) for each of the 10 categorical level of Cell.Shape.

Clearly, from the meaning of Cell.Shape there seems to be some sort of ordering within the categorical levels of Cell.Shape. That is, a cell shape value of 2 is greater than cell shape 1 and so on.

This is the case with other variables in the dataset a well. So, its preferable to convert them into numeric variables and remove the id column.

Had it been a pure categorical variable with no internal ordering, like, say the sex of the patient, you may leave that variable as a factor itself.

The logitmod is now built. You can now use it to predict the response on testData.

So if pred is greater than 0.5, it is malignant else it is benign.